Sentence Alignment in Parallel Corpora :

This report has two aims To give information about the issues behind corpus alignment and the techniques currently used. To describe a particular corpus which members of CCL were involved in constructing-the Asahi corpus. The subject of aligning parallel corpora is expanding rapidly, particularly because the bottom-up machine translation (MT) paradigms such as Example-based MT and Statistics-based MT are looking for large knowledge sources. However, most work has been done on aligning European language corpora such as the Canadian Hansard and this necessarily ignores many of the diicult issues we face when aligning more semantically distant languages such as English and Japanese. The Asahi corpus was constructed from a CD-ROM of newspaper editorials which were automatically aligned using a hybrid statistical-linguistic approach at the Nara Advanced Institute of Science and Technology (NAIST) in Japan. The Asahi editorials appear daily in a national, broadsheet newspaper and tend to comment on subjects in the news headlines. Although previous experiments on a small scale had shown NAIST's technique to be very reliable (less than 4 percent error), CCL researchers required an even smaller error rate and a ne tolerance for sentence alignment. Consequently we decided to check the corpus, some 330,000 words of English and a similar amount of Japanese, using a bilingual human. The report gives the statistical characteristics of the nal corpus and also a detailed subject breakdown.